Image-based 3D Object Detection for Autonomous Driving

Autonomous driving has the potential to radically change people’s lives, improving mobility and reducing travel time, energy consumption, and emissions. As one of the key enabling technologies for autonomous driving, image-based 3D object detection has received a lot of attention and gradually becomes a hot research topic. In this thesis, we review existing image-based 3D object detection models and propose novel taxonomies to help readers understand common pipelines in this area. We also create a simple baseline model to identify and address the key challenge: 'localization error.' In addition to the ‘result-lifting’ method, we introduce a successful 'pseudo-LiDAR' approach that outperforms other methods. We show that its effectiveness lies in coordinate transformation rather than data representation. We also show how to use LiDAR signals to guide the image-based models. Particularly, we propose a simple and effective scheme to introduce the spatial information from LiDAR signals to the monocular 3D detectors, without introducing any extra cost in the inference phase. We first transform the LiDAR signals into the image representation and train a LiDAR model with the same architecture as the baseline model. This LiDAR model can serve as the teacher to transfer the learned knowledge to the image model, and the experiments show the effectiveness of our scheme. Moreover, to leverage the massive unlabeled data, we also investigate how to apply image-based 3D detection in the semi-supervised setting with the help of LiDAR signals. In summary, in this thesis, we thoroughly review existing image-based 3D detection models and propose new image-based 3D detection paradigms with promising performances. Besides, we also show how to use auxiliary LiDAR signals to guide the image-based model learning spatial features and achieve semi-supervised learning. Finally, we discuss open questions in this research field and point out several promising research directions

Rethinking Pseudo-LiDAR Representation

The recently proposed pseudo-LiDAR based 3D detectors greatly improve the benchmark of monocular/stereo 3D detection task. However, the underlying mechanism remains obscure to the research community. In this paper, we perform an in-depth investigation and observe that the efficacy of pseudo-LiDAR representation comes from the coordinate transformation, instead of data representation itself. Based on this observation, we design an image based CNN detector named Patch-Net, which is more generalized and can be instantiated as pseudo-LiDAR based 3D detectors. Moreover, the pseudo-LiDAR data in our PatchNet is organized as the image representation, which means existing 2D CNN designs can be easily utilized for extracting deep features from input data and boosting 3D detection performance. We conduct extensive experiments on the challenging KITTI dataset, where the proposed PatchNet outperforms all existing pseudo-LiDAR based counterparts. Code has been made available at: https://github.com/xinzhuma/patchnet.Comment: ECCV2020. Supplemental Material attache

Transcriptional up-regulation of relaxin-3 by Nur77 attenuates β-adrenergic agonist-induced apoptosis in cardiomyocytes.

The relaxin family peptides have been shown to exert several beneficial effects on the heart, including anti-apoptosis, anti-fibrosis, and anti-hypertrophy activity. Understanding their regulation might provide new opportunities for therapeutic interventions, but the molecular mechanism(s) coordinating relaxin expression in the heart remain largely obscured. Previous work demonstrated a role for the orphan nuclear receptor Nur77 in regulating cardiomyocyte apoptosis. We therefore investigated Nur77 in the hopes of identifying novel relaxin regulators. Quantitative real-time PCR (qRT-PCR) and enzyme-linked immunosorbent assay (ELISA) data indicated that ectopic expression of orphan nuclear receptor Nur77 markedly increased the expression of latexin-3 (RLN3), but not relaxin-1 (RLN1), in neonatal rat ventricular cardiomyocytes (NRVMs). Furthermore, we found that the -adrenergic agonist isoproterenol (ISO) markedly stimulated RLN3 expression, and this stimulation was significantly attenuated in Nur77 knockdown cardiomyocytes and Nur77 knockout hearts. We showed that Nur77 significantly increased RLN3 promoter activity via specific binding to the RLN3 promoter, as demonstrated by electrophoretic mobility shift assay (EMSA) and chromatin immuno-precipitation (ChIP) assays. Furthermore, we found that Nur77 overexpression potently inhibited ISO-induced cardiomyocyte apoptosis, whereas this protective effect was significantly attenuated in RLN3 knockdown cardiomyocytes, suggesting that Nur77-induced RLN3 expression is an important mediator for the suppression of cardiomyocyte apoptosis. These findings show that Nur77 regulates RLN3 expression, therefore suppressing apoptosis in the heart, and suggest that activation of Nur77 may represent a useful therapeutic strategy for inhibition of cardiac fibrosis and heart failure. © 2018 You et al

GUPNet++: Geometry Uncertainty Propagation Network for Monocular 3D Object Detection

Geometry plays a significant role in monocular 3D object detection. It can be used to estimate object depth by using the perspective projection between object's physical size and 2D projection in the image plane, which can introduce mathematical priors into deep models. However, this projection process also introduces error amplification, where the error of the estimated height is amplified and reflected into the projected depth. It leads to unreliable depth inferences and also impairs training stability. To tackle this problem, we propose a novel Geometry Uncertainty Propagation Network (GUPNet++) by modeling geometry projection in a probabilistic manner. This ensures depth predictions are well-bounded and associated with a reasonable uncertainty. The significance of introducing such geometric uncertainty is two-fold: (1). It models the uncertainty propagation relationship of the geometry projection during training, improving the stability and efficiency of the end-to-end model learning. (2). It can be derived to a highly reliable confidence to indicate the quality of the 3D detection result, enabling more reliable detection inference. Experiments show that the proposed approach not only obtains (state-of-the-art) SOTA performance in image-based monocular 3D detection but also demonstrates superiority in efficacy with a simplified framework.Comment: 18 pages, 9 figure

High sensitivity ammonia gas sensor based on a silica gel coated microfiber coupler

In this paper, a high sensitivity ammonia gas sensor is proposed based on a silica gel coated microfiber coupler (MFC). The MFC structure is formed by the two tapered fibers with 3 μm waist diameter each, which were fabricated by using a customized microheater brushing technique. Silica gel coating was prepared by a sol-gel technique and applied on the surface of the MFC as a thin layer. The spectral characteristics of the proposed sensor were studied under various ammonia gas concentrations. The experimental results show that the coating thickness strongly affected the sensitivity of the MFC-based sensor to ammonia gas concentration. For the sensor with a 90 nm silica gel coating thickness, the highest measurement sensitivity is 2.23 nm/ppm for ammonia gas concentration, and the resolution is as good as 5 ppb, while the measured response and recovery times are ~ 50 and 35 seconds, respectively. Finally, it is demonstrated that the proposed sensor offers good repeatability and selectivity to ammonia gas

Microdisk Resonator With Negative Thermal Optical Coefficient Polymer for Refractive Index Sensing With Thermal Stability

In this paper, we propose a microdisk resonator with negative thermal optical coefficient (TOC) polymer for refractive index (RI) sensing with thermal stability. The transmission characteristics and sensing performances by using quasi-TE01 and quasi-TM01 modes are simulated by a three-dimensional finite element method. The influences of the TOC, RI, and thickness of the polymer on the sensing performances are also investigated. The simulation results show that the RI sensitivity Sn and temperature sensitivity ST with different polymers are in the ranges of 25.1-26 nm/RIU and 67.3-75.2 pm/K for the quasi-TE01 mode, and 94.5-110.6 nm/RIU and 1.2-51.3 pm/K for the quasi-TM01 mode, respectively. Moreover, figure-of-merit of the temperature sensing for the quasi-TM01 mode is in the range of 2 × 10-4-8 × 10-3, which can find important application in the implementation of the adiabatic devices

Microdisk Resonator With Negative Thermal Optical Coefficient Polymer for Refractive Index Sensing With Thermal Stability

In this paper, we propose a microdisk resonator with negative thermal optical coefficient (TOC) polymer for refractive index (RI) sensing with thermal stability. The transmission characteristics and sensing performances by using quasi-TE01 and quasi-TM01 modes are simulated by a three-dimensional finite element method. The influences of the TOC, RI, and thickness of the polymer on the sensing performances are also investigated. The simulation results show that the RI sensitivity Sn and temperature sensitivity ST with different polymers are in the ranges of 25.1-26 nm/RIU and 67.3-75.2 pm/K for the quasi-TE01 mode, and 94.5-110.6 nm/RIU and 1.2-51.3 pm/K for the quasi-TM01 mode, respectively. Moreover, figure-of-merit of the temperature sensing for the quasi-TM01 mode is in the range of 2 × 10 -4 -8 × 10 -3 , which can find important application in the implementation of the adiabatic devices